SMIL: Multimodal Learning with Severely Missing Modality

A common assumption in multimodal learning is the completeness of training data, i.e., full modalities are available all examples. Although there exists research endeavor developing novel methods to tackle incompleteness testing e.g., partially missing examples, few them can handle incomplete modalities. The problem becomes even more challenging if considering case severely missing, ninety percent examples may have For first time literature, this paper formally studies with modality terms flexibility (missing training, testing, or both) and efficiency (most data modality). Technically, we propose a new method named SMIL that leverages Bayesian meta-learning uniformly achieving both objectives. To validate our idea, conduct series experiments on three popular benchmarks: MM-IMDb, CMU-MOSI, avMNIST. results prove state-of-the-art performance over existing generative baselines including autoencoders adversarial networks.